Learning system with augmented reality and related learning method

ABSTRACT

A learning system with augmented reality is provided. The learning system includes a cloud server recording an operation history of a learner and providing feedback messages, and a mobile device having an image-capturing module capturing an image of a substantial object. Also, the learning system comprises an object database storing a simulated object corresponding to the substantial object, an identification module identifying the image and generating image information, and a processing module which receives and analyzes the image information, obtains the simulated object from the object database according to analyzing results, and displays the simulated object on a display interface of the mobile device. The learning system allows learner to operate simulated object operation instructions on the display interface or directly operate the substantial object to control a display status of the simulated object, and the operation history of learner is transmitted to the cloud server. Learner conducts simulation scientific experiments and substantial operational trainings by operating the simulated object or the substantial object, thereby facilitating learning abstract concepts without being limited by the time and space constraints in a conventional learning system.

BACKGROUND OF THE INVENTION

1. Field of the Invention

This invention relates to a combination application of mobiletechnology, cloud technology and augmented reality. The invention alsorelates more specifically to a learning system with augmented realityand a related learning method applicable to a mobile device to provide alearner with experimental operations.

2. Description of Related Art

With the rapid development of technology, a teaching task can berealized by books and tangible instructional tools. Through the use andassistance of the tangible instructional tool, a teacher's instructionwill not be limited by words and graphs and a learner can learn in amore intuitive and thorough way.

In a scientific world, some phenomena are invisible, such as microscopicparticles, and appear to be too abstract for beginning learners.Teachers typically teach students through books or computers, andstudents could only attempt to understand and imagine those phenomenaand their related concepts through the pictures and texts on paper orreal-time graphs displayed on the computer. However, if the concepts arerelated to complicated phenomena such as the moving of air particles,then it would be difficult for students to understand these conceptsthrough books and computer only. One feasible solution is to usemultimedia to facilitate learning. Yet, multimedia-based learningrequires a variety of software, and educational institutions oftencannot afford such high cost facilities. In the above-described learningprocess, both the teacher and the learners are faced with variousproblems. For example, the available media such as books and graphscould not provide sufficient scaffolding for learners to learn moreeffectively. Simultaneously, multimedia equipment is usually expensiveand unaffordable. Moreover, if the teacher intends to test the learners,paper is usually the only option since other testing methods, such as acomputer-based simulation test, are also expensive and constrained byspace and time.

Therefore, how to embody the abstract concepts in natural scientificphenomena to provide teachers and learners with a real-time, highlyefficient, and uncomplicated teaching, testing and experimental learningmechanism is becoming an urgent issue in the field.

SUMMARY OF THE INVENTION

In view of the above-mentioned problems, the present invention providesa learning system with augmented reality and a related learning method,which combine mobile technology, cloud technology and augmented realityand provide learning, testing and experimental operation learningmechanisms that present abstract concepts.

To achieve the above-mentioned and other educational objectives, thepresent invention provides a learning system with augmented reality,comprising a mobile device connected to a cloud server via network; themobile device would be provided to learner to operate and the cloudserver would record learner's operation history and provide feedbackmessages. The mobile device would be equipped with: an image-capturingmodule that can identify objects being used as instructional tools basedon a database that has stored various simulated objects and theircorresponding real-world objects; an identification module thatidentifies the image of the physical object that is captured by theimage-capturing module and generates image information; and a processingmodule that receives and analyzes the image information generated by theidentification module, obtains the simulated object corresponding to thesubstantial object from the object database according to theidentification pattern, and displays the simulated object on a displayinterface of the mobile device, wherein learner is allowed to operatesimulated object operation instructions on the display interface ordirectly operate the 3D object to control a display status of thesimulated object corresponding to the physical object, and learner'soperation history is transmitted back to the cloud server, and whereinlearner conducts simulation scientific experiments and interact with thesimulated object or the real-world 3D object.

In an embodiment, the mobile device also includes a communicationmodule, and the processing module would transmit the device's operationhistory via the communication module to the cloud server.

In an embodiment, the cloud server is made of the following parts: acomputation module that analyzes the operation history and generates thefeedback messages and history data; a history database that stores theoperation history and the history data generated by the computationmodule; a statistics module that gathers statistics of the history datain the history database and generates learning statistics data; and afeedback module that generates feedback instruction according to thefeedback messages generated by the computation module and the learningstatistics data generated by the statistics module, and transmits thefeedback instruction back to the processing module to provide real-timelearning feedbacks.

In an embodiment, learner is allowed to operate the substantial objectand capture a new image via the image-capturing module. According to thenew image, the display interface would display a new display status ofthe simulated object.

The present invention further provides a learning method with augmentedreality that allows a learner to conduct a learning process via a mobiledevice, comprising the following steps of: (1) providing a substantialobject used as an instructional tool and simulated objects correspondingto the substantial object; (2) setting an interaction relation and afeedback condition of the simulated objects; (3) capturing, by using themobile device, an image of the substantial object, identifying the imageof the substantial object and generating image information, andobtaining the simulated objects corresponding to the substantial objectaccording to the image information and displaying the simulated objectson a display interface of the mobile device; (4) controlling, by alearner, a display status of the simulated objects via simulated objectoperation instructions displayed on the display interface, or directlyoperating, by learner, the substantial object to control the simulatedobjects corresponding to the substantial object, and, recording, byusing the mobile device, an operation history of learner; and (5)automatically transmitting the operation history to a cloud server, andanalyzing, by the cloud server, the operation history and generatingfeedback messages for real-time feedbacks and history data for learningstatistics.

In an embodiment, the interactive relation and the feedback conditioncomprise an adjustable parameter, a space interactive relation, and anoperation history content ready to be recorded of the simulated objects.

Compared with conventional technique, the learning system with augmentedreality and related learning method according to the present inventionemploy a mobile device to capture an image of a substantialinstructional tool, and obtain and display a corresponding simulatedobject on the mobile device. A learner is allowed to directly operatethe substantial instructional tool or control the simulated object tochange a formed simulation image. Therefore, an experimental operationlearning effect is enhanced. The learning system is applicable tofacilitate teaching and testing, and the employment of a substantialinstructional tool achieves a demonstration of an abstract object orabstract concept and the learning of experimental operations. Thepresent invention employs the augmented reality techniques to presentand simulate the operations of an abstract concept with an intuitive andreal medium, so as to help a learner to understand and learn in aspatial interaction manner. With a mobile device, the learningrestrictions on time and space can be reduced. With the use of the cloudoperations, real-time feedbacks are provided, and learning histories ofa plurality of learners can be summarized, thus facilitating ourknowledge of learner's learning performance or adjustment of theteaching strategy for learner.

BRIEF DESCRIPTION OF DRAWINGS

The invention can be more fully understood by reading the followingdetailed description of the preferred embodiments, with reference madeto the accompanying drawings, wherein:

FIG. 1 is a functional block diagram of a learning system with augmentedreality of an embodiment according to the present invention;

FIG. 2 is a schematic diagram of a learning system with augmentedreality of an embodiment according to the present invention;

FIG. 3 is a functional block diagram schematically illustrating anoperation of a whole structure of a learning system with augmentedreality according to the present invention; and

FIG. 4 is a flow chart of a learning method with augmented reality of anembodiment according to the present invention.

DETAILED DESCRIPTION OF THE INVENTION

The following illustrative embodiments are provided to illustrate thedisclosure of the present invention, these and other advantages andeffects can be apparently understood by those in the art after readingthe disclosure of this specification. The present invention can also beperformed or applied by other different embodiments. The details of thespecification may be on the basis of different points and applications,and numerous modifications and variations can be devised withoutdeparting from the spirit of the present invention.

FIG. 1 is a functional block diagram of a learning system 1 withaugmented reality of an embodiment according to the present invention.The learning system 1 allows a learner to conduct an experimentaloperations via a mobile device 10.

Please note that the mobile device 10 is exemplary, and the learningsystem 1 according to the present invention is not limited thereto. Inan embodiment, the mobile device 10 is an intelligent mobile device. Inanother embodiment, the mobile device 10 comprises an image-capturingapparatus 106, such as a camera or a video camera, and a displayinterface 107 for displaying images. In an embodiment, the displayinterface 107 is an external projector. In another embodiment, thedisplay interface 107 is a display panel of the mobile device 10.

In an embodiment, the learning system 1 with augmented reality comprisesthe mobile device 10 for learner to operate, and a cloud server 12 thatprovides cloud service. The cloud server 12 records learner's operationhistory and provides appropriate feedback messages according tolearner's learning situations.

A learner can operate the mobile device 10. In addition to the embeddedimage-capturing apparatus 106 and the display interface 107, the mobiledevice 10 has internal processing units that conduct augmented realityand interactive learning. The internal processing unit includes animage-capturing module 101, an object database 102, an identificationmodule 103 and a processing module 104.

The image-capturing module 101 captures an image of a substantial object11 used as a instructional tool. In practice, the image-capturing module101 employs the image-capturing apparatus 106 in the mobile device 10 tocapture an external object image. The substantial object 11 represents ainstructional tool of an object, such as air molecules. Since the airmolecules are invisible, the substantial object 11, which has a specificshape or a special pattern, may be utilized to represent the airmolecules.

The object database 102 stores a simulated object and an identificationpattern corresponding to the substantial object 11. Since thesubstantial object 11 is designed for presenting an object that isinvisible, the substantial object 11 can have various identificationpatterns to define what the substantial object 11 is. For instance, thesubstantial object 11 (i.e., the instructional tool) is a 2D object withDNA pattern printed thereon, and learner can use such substantial object11 to present a simulated object related to DNA. Therefore, as long asthe simulated object is stored in the object database 102 in advance anda relation between the substantial object 11 and the simulated object isdefined accordingly, when the substantial object 11 is determined, acorresponding simulated object can be found for the presentation.Therefore, the object database 102 can store a plurality of simulatedobjects that correspond to a plurality of substantial objects 11. Theidentification pattern will be described in details in the followingparagraphs.

The identification module 103 identifies the image captured by theimage-capturing module 101 and generates image information. Theidentification module 103 receives and identifies, from theimage-capturing module 101, the image of the substantial object 11captured by the image-capturing apparatus 106. The identification module103 identifies what the substantial object 11 is.

The processing module 104 receives and analyzes the image informationfrom the identification module 103, and obtains, from the objectdatabase 102, the simulated object corresponding to the captured imageof the substantial object 11. In reality, the substantial object 11 isused as a instructional tool and corresponds to a simulated object, suchas air molecules. The processing module 104 then displays the simulatedobject on the display interface 107 of the mobile device 10.

In an embodiment, the substantial object 11 is a 2D substantial objector a 3D substantial object. In another embodiment, the substantialobject 11 can have a variety of shapes, as long as the identificationmodule 103 can identify what the substantial object 11 represents. Ofcourse, the substantial object 11 can have its shape be designed tocomply with various requirements, and can be adjusted according todesign demands of the instructional tool. In an embodiment, surfaces ofthe substantial object 11 have different patterns for identification,and the identification module 103 identifies an image formed by a singlesurface of the 2D substantial object or any surface of the 3Dsubstantial object.

Since the object database 102 stores pattern identification informationof every surface of the substantial object 11 in advance, theidentification module 103 can identify the image captured by theimage-capturing module 107, no matter how the substantial object 11 isdisposed or moved.

For instance, when the substantial object 11 is a 2D substantial objecthaving a specific pattern on a surface thereof, the identificationmodule 103 can identify what the substantial object 11 represents bysimply identifying the specific pattern. In another embodiment, in whichthe substantial object 11 is a 3D substantial object, such as ahexahedron, and the object database 102 is stored with a specificpattern of any surface of the hexahedron, after the identificationmodule 103 identifies the specific pattern of any surface of thesubstantial object (e.g., the hexahedron), what the substantial object11 represents is known.

In a learner's learning operation, learner is allowed to control astatus of the simulated object via simulated object operationinstructions on the display interface 107. In an embodiment, thesimulated object operation instructions indicate a control interface,such as a button, a slide bar or an option, or a graphical interfacesuch as a graph that can be shown on the display interface 107. Learneris allowed to touch or click the control interface or the graphicalinterface to trigger the operations of the simulated object. Forinstance, if the simulated object is air molecules, when a button thatincreases temperature is triggered, air molecules would move fasteraccording to a science principle, that is, as the temperature getshigher, the air molecules would be more active. In such a manner, thedisplay status of the simulated object can be controlled.

In an embodiment, learner is allowed to directly operate the substantialobject 11 to change the display status of the simulated object. Whenlearner changes the location or status of the substantial object 11, theimage-capturing module 101 captures a new image, and the displayinterface 107 displays a new display status of the simulated objectcorresponding to the new image. Hence, learner can move, rotate orcalibrate the substantial object 11 directly, and accordingly, thesimulated object in the display interface 107 changes correspondingly.

Therefore, with regard to a substantial object representing a simulatedmatter, learner is allowed to use the learning system 1 to generate adynamic and interactive simulated object, observe the display status ofthe simulated object, and change the display status by controlling thesimulated object or the substantial object via image capturing,analyzing and displaying processes. Therefore, learner can learnintuitively, without resorting to the descriptions of traditional textsand pictures.

In order to allow learner to obtain real-time feedbacks, or allow ateacher to know the learning situations of learner, and even allow theteacher to track the learning history of learner or gather statistics ofthe related learning history data, the learning process of learner istransmitted via the mobile device 10 to the cloud sever 12automatically.

In practice, the communication module 105 in the mobile device 10transmits the operation history generated by the processing module 104to the cloud server 12, and the cloud server 12 provides correspondingreal-time feedbacks with regard to different operation histories, orrecords the operation history of learner to further gather statistics oranalyzes several operation histories to transform the data intomeaningful data for further research and evaluation. In addition to thelearner's operation history, the communication module 105 furthertransmits the image captured by the image-capturing module 101 or thesimulated object from the object database 102 to the cloud server 12, soas to provide complete data.

In an embodiment, instead of transmitting the image or the simulatedobject directly to the cloud server 12, the communication module 105,when being operated, converts history information related to thecapturing of the image and the changing of the simulated object into ahistory record, and the history record is then transmitted to the cloudserver 12. For instance, how a learner, when facing a plurality ofsubstantial objects 11 numbered from A-Z, observes simulated objectsthat the substantial objects 11 represent, what the observation sequenceis, and when he observes each of the substantial objects 11 are to beunderstood. After identifying that the substantial object 11 and thesubstantial object 11 being identified is changing, the identificationmodule 103 uploads numerals having time stamps, so as to build a historyrecord of image identification. The communication module 105 can be usedas a medium for data transmission or message transmission.

The cloud server 12 comprises an computation module 121, a historydatabase 122, a statistics module 123 and a feedback module 124. Thecomputation module 121 analyzes an operation history from the mobiledevice 10 and generates feedback messages and history data. The historydatabase 122 stores the operation history received by the computationmodule 121 and the history data generated after analyzing the operationhistory by the computation module 121. The statistics module 123 gathersstatistics of the history data in the history database 122 and generateslearning statistics data from the history data. The feedback module 124generates feedback instructions according to the feedback messages ofthe computation module 121, and transmits the feedback instructions viathe communication module 105 of the mobile device 10 to the processingmodule 104, such that the display interface 107 or other mechanisms(vibration or sounds) would provide real-time learning feedbacks.Therefore, the cloud server 12 provides a complete record, integrates alearner's learning history, and provides feedbacks according to thelearning history.

In other words, the feedback module 124 can also generate feedbacks thatcan be shared by others according to the statistics module 123.Therefore, the learning system 1 can integrate many learners' learninghistories, this stands for the fact that scientific concept is extractedfrom many accumulated science experimental results. For instance, when ameasurement experiment operation is finished and the result is the sameas the statistics results of many learning histories, the feedbackinstructions are transmitted back, and the processing module 104 of themobile device 10 generates vibration and sounds, informing learner ofthe operation result. Moreover, the feedback module 124 can also displaythe integrated data in a page format on a web page, so as to share theintegrated data.

In an embodiment, the cloud server 12 can be used as a test recordserver. Therefore, learner can employ the learning system 1 to conduct atest, and the learner's operation history and test result will betransmitted back to the cloud server 12, for the cloud server 12 tofurther analyze and gather statistics. In the embodiment, a new testingmechanism is added to the learning system 1, and, as such, the learningsystem 1 can thus possess both teaching and testing functions. Suchlearning system 1 operates in a similar way to the previous ones, andthus further description is hereby omitted.

The operations of the learning system 1 will be described with anexample in the following description. Air particles are used as aconcept to be taught to illustrate the operations of the learning system1.

Please refer to FIGS. 1 and 2. FIG. 2 is a schematic diagram of alearning system 1 with augmented reality of an embodiment according tothe present invention. The learning system 1 allows a learner 13 toconduct an operation training of a science experiment. When conductinglearning or testing, the learner 13 can employ the image-capturingapparatus 106 of the mobile device 10 to capture an image of thesubstantial object 11. In an embodiment, the substantial object 11includes a 2D substantial object 111 and a 3D substantial object 112,which two have different patterns on surfaces thereof. After theimage-capturing apparatus 106 captures images of the 2D substantialobject 111 and the 3D substantial object 112, the object database 102,after analyzing the images, finds a corresponding simulated object anddisplays the simulated object on the display interface 107. In anembodiment in which air particles are used as an example, the mainobjective is to teach the movement of the air particles. The 2Dsubstantial object 111 indicates a glass box, and the 3D substantialobject 112 represents the air particles. The display interface 107displays simulated objects 14 including the 3D air particles 141 and the3D glass box 142. A learner is allowed to control a status of thesimulated objects 14 via the simulated object operation instructions1071 on the display interface 107.

With regard to the determination of the image of the substantial object11, when the learner 13 is capturing the image of the substantial object11 with the image-capturing apparatus 106 of the mobile device 10,wherein the 2D substantial object 111 has an identification pattern 111′and the 3D substantial object 112 has an identification pattern 112′,the identification module 103 analyzes the images according to anidentification pattern set 11′ of the identification patterns 111′ and112′. Since different patterns represent different simulated objects,the processing module 104 can find in the object database 102corresponding simulated objects 14, i.e., the 3D air particles 141 andthe 3D glass box 142. Each of the simulated objects 14 has its owndisplay status information, and the learner 13 is allowed to observe viathe display interface 107 that the 3D air particles 141 are trapped inthe 3D glass box 142, and move, collide and bounce. Therefore, aninvisible microcosmic science phenomenon is embodied. Moreover, thelearner 13 is allowed to control the status of the simulated objects 14via the simulated object operation instructions 1071, e.g., byincreasing the temperature or number of the air particles, thus causingthe simulated air particles to move in a high speed and collide in ahigh frequency, so as to achieve the objective of intuitive learning.

The learner 13 is also allowed to conduct experimental operations bymanually operating the substantial object 11. For instance, asubstantial object (not shown) corresponding to a 3D transparent wall isadded. After the substantial object is identified, augmented reality ofthe 3D transparent wall (not shown) is generated. When the learner 13manually presses the 3D transparent wall into the 3D glass box 142 (the3D transparent wall and the 3D glass box 142 can pass through eachother), the movement space of the simulated 3D air particles 141 arecompressed, and the frequency of collision of the air particles 141 inthe simulated 3D glass box 142 is increased, which represents that thepressure in the 3D glass box 142 is increased. Therefore, through theabove simulation experiment operations, the learner 13 is allowed todirectly operate the substantial object 11, and to achieve the effect ofexperimental operation training. The identification pattern set 11′, thesimulated object 14 and the process that the learner 13 executes thesimulated object operation instructions 1071 will be recorded andtransmitted to the history database 122 of the cloud server 12, forfurther researches.

FIG. 3 is a functional block diagram of a learning system 1 withaugmented reality according to the present invention. The learningsystem 1 allows the learner 13 to conduct operation training of scienceexperiments. An image-capturing apparatus (e.g., a video camera)installed in the mobile device 10 captures an image of the substantialobject 11, and analyzes the captured image to find a correspondingsimulated object 14. The simulated object 14 is displayed on a monitor,for the learner 13 to observe and learn.

When learning the simulated object 14, the learner 13 can touch andcontrol a button, a slide bar or an option on an operation interface ofthe mobile device 10, to control a status of the simulated object 14 viaa parameter control 53. Alternatively, the learner 13 can hold, move andcontrol the substantial object 11 in a space interaction 52 manner, suchas moving, rotating and calibrating actions to operate the substantialobject 11. In other words, the learner 13 is allowed to directly controlthe substantial object 11 in a real space to change the simulated object14 indirectly, so as to achieve the objective of interaction. Inaddition, in order to record the learner's 13 learning controllingactions, sensors in the mobile device that sense direction,acceleration, brightness and distance can be used to sense the learner's13 controlling process on the mobile device 10.

During the interaction process, the space interaction 52 and theparameter control 53 conducted by the learner 13 and the status changegenerated by the simulated object 14 generate space movement information510, parameter adjustment information 511 and simulated interactioninformation 512, respectively. The space movement information 510represents 3D coordinate information of the simulated objectsuccessively transmitted when the corresponding substantial object 11 isidentified successfully. The parameter adjustment information 511 is astatus value of the simulated object 14 that corresponds to an operationinterface. The simulated interaction information 512 is results of aninteraction relation generated by a certain design including contact(e.g., the surfaces of the simulated objects collide), combination(e.g., the simulated objects 14 engage with respect to shapes orcorrespond with respect to spaces), and overlap (e.g., the spaces of thesimulated objects are stacked on one another) of the simulated objects14, rather than the information related to the change of the simulatedobject 14 itself. The above data is recorded in an automaticallyrecording history 54 process as three history data 513 having a timedirection that are generated by automatically providing time stamps toinformation generated by an interaction process and integrating theinformation. In the embodiment, in which the 3D transparent wall isadded to the experimental operation, space movement information 510 ofthe 3D transparent wall is generated, the collision frequency isincreased because the space in which the air particles move is reduced,and the simulated interaction information 512 that has collisionfrequency changed is generated. The above two information and theparameter adjustment information 511 that is generated when theparameter control is conducted are integrated as the history data 513.

The history data 513 can be transmitted wirelessly to the remote cloudserver 12, and the cloud server 12 can conduct applications, such as adevice real-time feedback 55 and a network real-time sharing 56. Thedevice real-time feedback 55 indicates that the learner 13 is feedbackby vibration, sounds and animation effects and informed of his learningsituation and his comparison result with other learners. The networkreal-time sharing 56 indicates updating the history data 513 of aplurality of mobile devices 10 (i.e., of a plurality of learners) viaweb pages or mails, in order to provide a learning grade to the learner13, and allow the teacher to gather statistics and conduct quantitativeresearch.

The present invention creates a new-type teaching and testing system byintegrating a variety of technology concepts, such as intelligentmobility, augmented reality and cloud operations. Abstract scienceconcepts and invisible microcosmic phenomena (e.g., air particlesmovement) can thus be embodied and controlled with the aid ofexperiments, which helps learner to learn, reduces the cost, and bringsmore convenience. By using the cloud operations to record the historydata, the objectives of network real-time sharing and analysisstatistics can be achieved, which can be employed in research andanalysis.

FIG. 4 is a flow chart of a learning method with augmented reality of anembodiment according to the present invention. The learning method isembodied through the use of a mobile device. In step S401, a substantialobject used as a instructional tool and simulated objects correspondingto the substantial object are provided. In an embodiment, step S401further comprises fabricating the simulated objects and anidentification pattern, wherein the simulated objects are a 2D or 3Dcomputer graphs, animation or films, the identification pattern is ablack-white or highly-complicated color natural image, a correspondingrelation of the simulated objects and the identification pattern isdefined, the simulated objects can display dynamic or static displaystatuses, and the simulated objects and the identification pattern willbe recorded in an object database.

In an embodiment, the substantial object fabricated to be theinstructional tool comprises a 2D substantial object and a 3Dsubstantial object, both of which comprise identification patterns onsurfaces thereof, as a basis for the simulated objects to form an image.The method proceeds to step S402.

In step S402, an interaction relation and a feedback condition among thesimulated objects are set. In other words, the interaction relationamong the simulated objects and the feedbacks provided under differentoperations are set in advance. In an embodiment, an adjustable parameterand a space interaction relation are included, the adjustable parameterincludes a natural status, such as temperature, pressure and density andcan be operated by controlling options on a display interface, and thespace interaction relation indicates the relation of the simulatedobjects, such as the facts that simulated air particles collide or areenclosed by a simulated glass box. The method proceeds to step S403.

In step S403, the mobile device captures an image of the substantialobject, identifies the image of the substantial object and generatesimage information, obtains the simulated objects corresponding to thesubstantial object according to the image information, and displays thesimulated objects on a display interface of the mobile device. Learneris allowed to capture an image of the substantial object with theimage-capturing apparatus of the mobile device. The image can beidentified according to the identification patterns on a single surfaceof the 2D substantial object or any surface of the 3D substantialobject.

The image, after being captured, can be identified and analyzed, i.e.,identifying what the pattern on the surface of the substantial objectis, to obtain a simulated image corresponding to the substantial object.The simulated objects are displayed on the display interface of themobile device. Learner is thus allowed to observe the image of thesimulated objects on the display interface. The method proceeds to stepS404.

In step S404, the learner controls the display status of the simulatedobjects via the simulated object operation instructions on the displayinterface, or controls the simulated objects corresponding to thesubstantial object by directly operating the substantial object. Themobile device records the learner's operation history. Step S404illustrates that the simulated objects are displayed on the displayinterface of the mobile device, and the display status of the simulatedobjects can be controlled via adjustable parameters that are installedin advance to control the display statuses or interaction conditions ofthe simulated objects. The corresponding display statuses and theinteraction conditions are also recorded as the learner's operationhistory.

The learner is allowed to control the display status of the simulatedobjects via a visualized operation interface on the display interfacesuch as simulated object operation instructions, or change the status ofthe substantial object directly by changing the status of thesubstantial object in a real space by moving, rotating and calibratingmanners. The display interface will display a new status as the statusof the simulated objects are changed. Accordingly, the mobile device,when operating in an image-capturing mode, will keep capturing the imageof the substantial object, and keep identifying whether the imageexists, so as to keep updating the position of the simulated objectscorresponding to the substantial object in the real space. In anembodiment, the mobile device has hardware that keeps capturing imagesin a high speed, and software that keeps performing a redrawing process.The method proceeds to step S405.

In step S405, the operation history is transmitted to a cloud serverautomatically. The cloud server analyzes the operation history andgenerates feedback messages for real-time feedbacks and history data forlearning statistics. In order to achieve the objectives of real-timefeedbacks and data analysis statistics, the learning method according tothe present invention transmits the operation history to the cloudserver, and the cloud server analyzes the operation history andgenerates feedback messages and history data. As described previously,the learner's operation history can be recorded and stored in the cloudserver. In addition to providing real-time feedbacks and interaction,the data can be analyzed to gather statistics as a basis for improvingteaching or testing processes.

A learning system with augmented reality and a related learning methodaccording to the present invention embody an invisible teaching contentvia an augmented reality mechanism, so as to achieve a better learningeffect. In particular, the present invention employs augmented realityto focus on a learner's experimental operations, i.e., conductingsimulated experimental operation trainings with augmented reality as amedium, which is different from the teaching contents provided by books,graphs or films. The learning system according to the present inventionfurther provides testing, and frees learner from the limitation of theconventional paper-based tests, such that learner can obtain thereal-time feedback testing result, which is beneficial to the teachingand testing.

The foregoing descriptions of the detailed embodiments are onlyillustrated to disclose the features and functions of the presentinvention and not restrictive of the scope of the present invention. Itshould be understood to those in the art that all modifications andvariations according to the spirit and principle in the disclosure ofthe present invention should fall within the scope of the appendedclaims.

What is claimed is:
 1. A learning system with augmented reality,comprising: a cloud server that records a learner's operation historyand provides feedback messages; and a mobile device for the learner tooperate, comprising: an image-capturing module that captures an image ofa substantial object used as an instructional tool; an object databasethat stores a simulated object and an identification patterncorresponding to the substantial object; an identification module thatidentifies the image of the substantial object that is captured by theimage-capturing module and generates image information; and a processingmodule that receives and analyzes the image information generated by theidentification module, obtains the simulated object corresponding to thesubstantial object from the object database according to theidentification pattern, and displays the simulated object on a displayinterface of the mobile device, wherein the learner is allowed tooperate simulated object operation instructions on the display interfaceor directly operate the substantial object to control a display statusof the simulated object corresponding to the substantial object, and thelearner's operation history is transmitted back to the cloud server, andwherein the learner conducts simulation science experiments andsubstantial operational trainings by operating the simulated object orthe substantial object.
 2. The learning system of claim 1, wherein thesubstantial object comprises a 2D substantial object and a 3Dsubstantial object, and the identification module identifies an imageformed by a single surface of the 2D substantial object or any surfaceof the 3D substantial object.
 3. The learning system of claim 1, whereinthe mobile device further comprises a communication module, and theprocessing module transmits the operation history via the communicationmodule to the cloud server.
 4. The learning system of claim 3, whereinthe cloud server comprises: an computation module that analyzes theoperation history and generates the feedback messages and history data;a history database that stores the operation history and the historydata generated by the computation module; a statistics module thatgathers statistics of the history data in the history database andgenerates learning statistics data; and a feedback module that generatesa feedback instruction according to the feedback messages generated bythe computation module and the learning statistics data generated by thestatistics module, and transmits the feedback instruction back to theprocessing module for providing real-time learning feedbacks.
 5. Thelearning system of claim 1, wherein the simulated object operationinstructions comprise a button, a slide bar or an option to control thedisplay status of the simulated object.
 6. The learning system of claim1, wherein when the learner is operating the substantial object, theimage-capturing module captures a new image, and the display interfacedisplays a new status of the simulated object according to the newimage.
 7. A learning method with augmented reality that allows a learnerto conduct a learning process via a mobile device, comprising thefollowing steps of: (1) providing a substantial object used as ainstructional tool and simulated objects corresponding to thesubstantial object; (2) setting an interaction relation and a feedbackcondition of the simulated objects; (3) capturing, by using the mobiledevice, an image of the substantial object, identifying the image of thesubstantial object and generating image information, and obtaining thesimulated objects corresponding to the substantial object according tothe image information and displaying the simulated objects on a displayinterface of the mobile device; (4) controlling, by a learner, a displaystatus of the simulated objects via simulated object operationinstructions on the display interface, or directly operating, by thelearner, the substantial object, to control the simulated objectscorresponding to the substantial object, and recording, by using themobile device, an operation history of the learner; and (5)automatically transmitting the operation history to a cloud server, andanalyzing, by the cloud server, the operation history and generatingfeedback messages for real-time feedbacks and history data for learningstatistics.
 8. The learning method of claim 7, wherein the substantialobject comprises a 2D substantial object and a 3D substantial object,and the step (3) further comprises identifying an image formed by asingle surface of the 2D substantial object or any surface of the 3Dsubstantial object.
 9. The learning method of claim 7, wherein the step(4) further comprises capturing a new image according to changing, madeby the learner, a status of the substantial object, and displaying, byusing the display interface, a new status of the simulated objectsaccording to the new image and a space interaction status with othersimulated objects.
 10. The learning method of claim 7, wherein theinteraction relation and the feedback condition comprise an adjustableparameter, a space interaction relation, and an operation historycontent ready to be recorded of the simulated object.